Information literacy standards and the World Wide Web: results from a student survey on... Page 1 of 33 


/ftinformationresearch 

VOL. 19 NO. 4, DECEMBER, 2014 


• Contents | 

• Author index 

• Subject index 

• Search | 

• Home 


Information literacy standards and the World Wide Web: 
results from a student survey on evaluation of Internet 
information sources 


Arthur Taylor 

Information Systems and Supply Chain Management, College of Business 
Administration, Rider University, Lawrenceville, New Jersey, USA 

Heather A. Dalai 

Moore Library, Rider University, Lawrenceville, New Jersey, USA 


Abstract 

Introduction. This paper aims to determine how appropriate information literacy 
instruction is for preparing students for these unmediated searches using commercial 
search engines and the Web. 

Method. A survey was designed using the 2000 Association of College and Research 
Libraries literacy competency standards for higher education. Survey questions 
examined how subjects perceived the source evaluation criteria of reliability, validity, 
accuracy, authority, timeliness and point of view or bias. 

Analysis. Quantitative analysis was carried out on the data collected from 389 survey 
respondents who answered twenty-seven multiple choice questions concerning their 
information search practice and their evaluations of information sources. 

Results. Subjects primarily use Google as a research source for academic work and 
appear to be confused about how to determine the author of a source and how to 
determine the qualifications of the author. About half of the subjects indicated they may 
not be able to detennine the author of an Internet source yet consider it possible to 
determine the objectivity of the source. 

Conclusions. Information literacy instruction on source evaluation criteria may need to 
be reexamined in relation to the various information sources available today. More 
effective information literacy instruction methods which address the issues identified in 
this study may need to be implemented. 
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CHANGE FONT 


Introduction 

It is kn own that some significant portion of college research is done using commercial search engines 
on the Web. In this process, traditional gatekeepers of research have been replaced by search engine 
applications with u nkn own algorithms which return content of dubious quality. That college students 
today use the Internet for research is not unexpected. Most have had constant exposure to the Web 
and its vast sea of fragmented and unstructured information. Infonnation literacy instruction attempts 
to address some of these issues. Librarians, however, often have difficulties scheduling any 
information literacy instruction and when they can perform the instruction, they have a limited 
amount of time to discuss evaluating sources. 

Librarians and instructors strive to teach college students to evaluate sources of information for all 
their research projects. Critically evaluating information sources is a life skill and considered a core 
competency of information literacy highlighted by the Association of College and Research Libraries 
Information Literacy Competency Standard 3 (Association of College and Research Libraries, 2000) . 
The standards have given librarians the support needed to advocate for information literacy, but 
librarians with just a one-time guest instructional session can do little more than suggest students 
evaluate Web sources and highlight the criteria to use, often in a checklist form. 

It is unclear how effective information literacy instruction has been in teaching students to specifically 
evaluate Web information sources. A better understanding of how students perceive infonnation 
sources on the Web and how they evaluate the suitability of those sources for their research could 
help improve information literacy instruction. 

The intention of the research reported here was to evaluate the student perceptions of Web 
information sources, specifically their evaluation of various properties of those information sources in 
relation to traditional information literacy instruction. The study investigated student understanding of 
the evaluative criteria recommended in the Association of College and Research Libraries Information 
Literacy Competency Standard Performance Indicator 3.2a concerning "reliability, validity, accuracy, 
authority, timeliness, and point of view or bias" of sources (Association of College and Research 
Libraries, 2000) . (See Appendix B for the Association of College and Research Libraries standards, 
perfonnance indicator and outcomes that informed this research). 

Literature review 

Traditionally, it has been the librarian’s role to ensure students have access to quality material. While 
most librarians and faculty direct students to quality information sources, students easily bypass the 
traditional method and use familiar search engines. A gap exists between what sources professors 
expect students to cite in their academic research papers and the sources students actually use. Grimes 
and Boening (2001) identify two reasons why the gap might exist. In their study they report that 
faculty provided instruction which directed students to use reliable resources. Results of their study 
indicated that faculty felt their advice on evaluating information sources was ignored, and students 
indicated that they did not know what was meant by a reliable resource. More confusion exists about 
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what is acceptable to the instructor versus what librarians recommend, yet it is the librarians who are 
commonly tasked with ensuring students meet information literacy outcomes (Weiner, 2014) . 

Librarians lament when their information literacy efforts go to waste. Students show they have not 
retained knowledge when they cite unacceptable sources in their academic research papers. In 
attempts to improve information literacy instruction, librarians often get creative. Librarians might 
have an entire hour or just ten minutes to devote to teaching the evaluation of sources and strive to 
deliver effective instruction to leave students prepared to evaluate any infonnation source they 
encounter. Evaluation of sources as an information literacy competency standard has become 
increasingly important as information on the Web continues to expand dramatically. 

To address students' gaps in information literacy skills, many different approaches to Website 
evaluation have been taught by librarians. Librarians often teach to a checklist model with various 
acronyms and mnemonic devices such as RADCAB (relevancy, appropriateness, detail, currency, 
authority, bias), and CRAAP (currency, relevance, authority, accuracy, purpose) (Christennson, 2013 ; 
Blakeslee, 2004 ; Grimes and Boening, 2001 ; Shanahan, 2008) . The goal of the checklists is to address 
the criteria defined in Association of College and Research Libraries information literacy competency 
performance outcome 3.2a, which stresses the importance of evaluating source research documents 
(Meola, 2004 ; Metzger, 2007) . Librarians' information literacy instruction may also provide examples 
showcasing pre-selected Websites that might be good, bad, extremist or a hoax (Mathson and 
Lorezen, 2008) . Other librarians' information literacy instruction may vary from the standard 
approach to information literacy instruction to teaching using various learning styles (Dahlen, 2012) , 
teaching to various stages in cognitive development (Jackson, 2007: Benies-Small, Archer, Tucker, 
Vassady and Resor , 2013) , teaching using a contextual approach (Meola, 2004) or a constructivist 
approach (O'Connor et al ., 2010) , and engaging in new strategies such as active learning and 
redesigning learning spaces ( Detlor, Booker, Serenko and Julien, 2013; Hsieh, Hofmann, Dawson and 
Titus, 2014; Julian, 2013) . 

Existing literature also discusses partnerships between librarians and faculty. Writing instructors and 
librarians partner in teaching writing information literacy as both have similar goals ( O'Connor et al ., 
2010; Norgaard, 2004) . There are many successful librarian and discipline faculty collaborations 
which can be found in the literature, for example, in engineering (Wang, 2011) , nursing (Dee and 
Reynolds, 2013) , business (Atwong and Heichman, 2008) and other domains (Weiner, 2014) . 

Ultimately it is the student who must apply the criteria and perfonn the evaluation of information 
sources. Studies however have reported a discrepancy between what students say they can do and how 
they actually perform. Students can confidently articulate what to do or what to look for in evaluating 
sources, but when they have to perform research, they either do not know how to do it, do not have 
the skills or just do not perform the work needed to evaluate sources correctly. Thus, the criteria 
students say they will use are not the same criteria they actually use (Currie et ah, 2010 ; Grimes and 
Boening, 2001 ; Walraven, Brand-Gruwel and Boshuizen, 2009) . A part of this discrepancy could be 
linked to students not really grasping how to apply the evaluative criteria or not understanding the 
need to apply the criteria while examining sources. 

Other problems in the use of evaluative criteria are also evident. In Grimes and Boening (2001) some 
students assigned credibility based on how a page looked and students ranked the author as the most 
important feature of a quality Website yet failed to consider the author's qualifications (ibid, p. 18). 
Studies have also provided some indication that students claim to evaluate their sources much more 
than they do in reality (Flanagin and Metzger, 2008 ; Gross and Latham, 2011 ; Metzger, Flanagin and 
Zwarun, 2003) . Gross and Latham (2011) recognise that this is part of a greater pedagogical problem 
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in which students often believe they have skills to find, evaluate and use information, but teaching 
students to be more self-reflective of their need to learn without damaging their self-efficacy is 
challenging. 

Previous research has also provided some indication that college-age searchers tend to skim sources 
and use just one or two pages of content to make a determination about the quality or suitability of a 
potential research source (Rowlands et al„ 2008) . Students also admit to selecting a source because 
their search keywords were in the title or in one of the lead paragraphs of a document ( Currie el ai, 
2010) . When probed, students disclose that their focus is on finding an answer and that the source of 
the information is of little concern (Van Deursen and Van Diepen, 2013) . Research suggests that this 
light review of content may be related to disinterest or lack of concern for the evaluation of various 
characteristics of content among college-age searchers and Internet users in general (Williams and 
Rowlands, 2007; Hirsh, 1999 ; Grimes and Boening, 2001 ; Lorenzen, 2001 ; Carr, 2010) . It may be 
that in viewing the search process as just another task, a consumerist approach may prevail whereby 
searchers simply view infonnation as just another commodity to be consumed at the lowest possible 
cost, identifying cost in this case as ease-of-access with limited consideration for quality (Buczynski, 
2005; Thompson, 2003 ; Young and Von Segern, 2001) . 

Some research has also focused specifically on the Web as a research source. Studies have suggested 
that students often prefer Google as their first choice for research (Head and Eisenberg, 2010) and that 
Web materials are easier for students to find (Biddix, Chung and Park, 2011; Currie, Devlin, Emde 
and Graves, 2010) . Studies have also examined how students evaluate documents found through 
various information retrieval sources. In Taylor (2012), subjects appeared to have limited interest in 
source quality and appeared to procrastinate in their effort to find information as the deadline for the 
research project approached. 

Other studies have identified additional information literacy problems specific to Web sources. 
Thirty-eight percent of students in a study reported by Tillotson (2002 ) did not recognise the need for 
evaluating resources, indicating that they rarefy find incorrect information on the Web. Students are 
aware that not all information on the Web is reliable but appear to forget this while searching. 

Students report trusting institutional Web pages, Web pages free from spelling errors, Web pages with 
a list of good references, a professional layout, easy navigation, and no commercial purpose and/or a 
lot of detail (Lorenzen, 2001 ; Hilligoss and Rieh, 2008; Head and Eisenberg, 2010) . 

To understand how students evaluate sources, a clearer understanding of the criteria they use to 
evaluate those sources is needed. In a study conducted by Project Information Literacy, students rated 
how often evaluative criteria (currency, author's credentials and different viewpoints expressed) 
affected their selection of research sources. Questions on the survey instrument used inquired about 
which parts of the course-related research process were difficult. The survey included question 
categories for filtering irrelevant results, 'determining credibility', and 'evaluating sources', but 
questions did not specifically enquire about difficulties in determining page authorship or other 
evaluative criteria (Head and Eisenberg, 2010 , p.25). Other surveys have examined social cues, social 
networks and students' use of instructors, classmates, friends/family, librarians and others for help 
with evaluation of information sources, but specific criteria used for evaluations were not examined 
(Metzger, Flanagin and Medders, 2010 ; Head and Eisenberg, 2010 , pp. 13, 40). 

In order the better to teach information literacy, librarians need to improve their understanding of 
students' decision processes when evaluating and selecting a source for their research assignments. If, 
as some studies suggest, the Web has become a primary source of information for student researchers, 
then a better understanding of how students specifically evaluate Web information sources is needed. 
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Previous research has not specifically examined the criteria for source evaluation, such as source 
authorship, authority, accuracy, credibility, objectivity and quality, by the student information seeker. 
Research which examines these aspects of site evaluation for Web sources is needed. Such research 
will help clarify educators' understanding of these aspects of student information gathering and 
provide suggestions for improving information literacy instruction. 

Research design 

To understand better how searchers evaluate information sources and to examine this evaluation 
process in relation to current information literacy instructional strategies, the following research 
questions were examined in this study. 

In relation to the source evaluation criteria identified by the information literacy competency 
standards (reliability, validity, accuracy, authority, timeliness and point of view or bias): 

• Do searchers consider it possible to determine these criteria for documents being evaluated? 

• How do searchers attempt to evaluate these criteria when evaluating information sources? 

To gather information on the evaluation of sources during the information search process, a survey 
instrument was developed (see Appendix A) . The survey questions were based on the Association of 
College and Research Libraries information literacy competency standards for higher education 
published in 2000. Questions were based primarily on standard three from this publication which 
indicates that 'the information literate student evaluates information and its sources critically and 
incorporates selected information into his or her knowledge base and value system' (Association of 
College and Research Libraries, 2000). A small number of questions were also used to gather 
demographic and other information about the subject. (See Appendix B for a specific description of 
how questions were mapped to the Association of College and Research Libraries information literacy 
competency standards.) 

The survey instrument was designed to be delivered online, through the Web. Students were not 
monitored as they completed the survey and were allowed to take the survey at their convenience. 
Subjects took the survey before receiving information literacy instruction from library faculty at the 
university. Survey results were stored anonymously in a database for later analysis. 

Subjects taking the survey were presented with a screen which indicated that the research was 
voluntary and that they could stop at any time. They were then presented with instructions on how to 
complete the survey (see Figure 1). Subjects were directed to answer questions about how they 
conducted and evaluated sources of information for their college research across all college courses. 

Most subjects completed the twenty-seven questions in the survey in between five and ten minutes. 
The data collected was anonymous. Subjects were not directly compensated for participation, but 
some subjects were allowed to optionally be entered into a pool to win a $50 gift certificate. 



Figure 1: Online survey instrument 
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The survey taken by the subjects contained twenty-seven questions (see Appendix A). (Note that 
question twenty-eight on the survey allowed subjects to optionally enter their email to be entered in a 
pool for a gift certificate and therefore was not analysed.) Respondents provided demographic and 
background information for questions one to eight. Respondents provided answers relating to how 
they searched for infonnation and evaluated information sources for questions nine to twenty-seven. 
Survey questions nine to twenty-seven were related to Association of College and Research Libraries 
infonnation literacy standards (2000) as detailed in Appendix B. 

Results 

The 389 subjects who participated were undergraduate students at an American university. Most were 
business majors (38%) with the remainder being a mix of liberal arts, education and communication 
majors. Most subjects were female (66%). Most subjects indicated they had a grade point average 
(GPA) of greater than 3.0 (74%). 

(Note that for all survey questions presented in the tables that follow, the percentage column is 
calculated as the number of respondents who selected the item divided by the total number of 
respondents for the survey. Using this approach for single select questions, the sum of the percentages 
for a question response will be equal to or close to 100% with some deviations due to rounding errors. 
Using this same approach for multiple select questions, the sum of the percentages for a question 
response will not total 100% for a question.) 

Information search habits 

When asked about their search habits, approximately 90% of the subjects indicated they searched for 
infonnation on the Web several times a day. Subjects also indicated they were most likely to search 
using Google, but suggested that other search engines and tools were also used in their research (see 
Table 1). Twenty-five percent of the subjects indicated they rarely use a search engine other than 
Google, 5% indicate they never use a search engine beyond Google, and 23% indicated they 
infrequently (less than 25% of the time) use a search engine beyond Google, so for 53% (25+23+5) of 
our sample, Google is the most commonly used information search tool for academic research. 


Table 1: Question 10 - Consider the papers that you have 
written in the last year. How often have you used research 
tools beyond Google or Ask.com to do your research? 


Response text Percentage 

Never 6% 

Infrequently - less than 25 per cent of the time. 23% 
Usually - about 75 per cent of the time. 20% 

Almost always. >26% 


When asked how they identity whether or not to use a source in their research, subjects provided a 
number of different answers from a set of criteria as shown in Table 2. The most common selection 
was that the Website was understandable and current, suggesting that Websites which were difficult 
to understand or did not appear to have current information were not used. It is interesting to note that 
the criteria for credentials of the author and the amount of information were selected less commonly 
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than the criteria of understandability and currency of the information, suggesting that the credentials 
of the author were less of a concern for some of the subjects. 


Table 2: Question 9 - When a search engine returns a list of 
pages, I select a page from the list based on the following 
criteria. Check all that apply. 


Response text 

Percentage 

the format of the site 

57% 

whether or not the site has video 

5% 

the site has pictures 

12% 

the site has a lot of information 

54% 

the site is understandable 

70% 

I am familiar with the author of the site 

22% 

I can verify the information on the site 

62% 

the credentials (qualifications) of the author are 
good 

54% 

the site is current 

70% 

the quality of the writing on the site is good 

60% 

there are reviews and comments on the site 

22% 


Evaluation of author qualifications and site authority 

Subjects provided a range of answers concerning the authority of a Web page and the qualifications of 
the author. When asked how they determine the author of a Web page (see Table 3), almost half of the 
respondents indicated they could not make such a detennination (45%) with a little over half 
indicating they could (55%). A similar conundrum exists with evaluating the qualifications of the 
author of the Web page. When asked if they could determine the qualifications of the author of a Web 
page, only 30% indicated they could, 63% of the respondents indicated they could not, and 7% 
indicated they did not understand what was meant by qualifications in the question. These results 
provide some indication of confusion concerning this infonnation literacy instruction requirement and 
perhaps an issue with Web content in general. 

A similar question asked subjects about evaluating whether or not a site is authoritative. A little over 
half indicated they could detennine whether or not a site was authoritative (53%), almost a quarter 
(24%) indicated they specifically could not determine the authoritativeness of a site, and almost a 
quarter (23%) indicated they did not understand (perhaps hedging because they do not check or do not 
know how to check). 


Table 3: TRUE/FALSE questions on information source characteristics 


Question 


True False 


Don't 

understand 


It is usually possible to determine the author of a Web page. 

It is usually possible to determine the qualifications of the author of a 
Web page. 


55% 45% 
30% 63% 


7% 
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It is usually possible to determine whether or not a site is authoritative. 
It is usually possible to determine whether or not a Web page is 
objective and unbiased. 

It is usually possible to determine the credibility of a Web page. 

I believe the pages listed by a search engine results usually contain 
accurate information. 


53% 24% 
74% 25% 
80% 20% 
60% 39% 


23% 

1 % 


In examining how the subjects evaluated the authority of a site (see Table 4), subjects appeared to be 
confused by the general concept of authority, with 17% of respondents indicating they did not 
understand what was meant by that term in relation to Websites. Ten percent of respondents do not 
evaluate the authority of a site and 6% do not believe it is possible to examine the authority of a site. 
The total of these three selections represent one third (33%) of respondents who do not evaluate the 
authority of a site for some reason, whether it is lack of understanding or the belief that it is not 
possible to examine the authority of a site. Thirty-nine percent of respondents use the URL of the site 
to determine authority (perhaps not the best indicator of authority). Twelve percent confer with 
someone knowledgeable in the subject domain and 43% check to see if the author has published other 
content. 


Table 4: Question 24: How do you evaluate the authority of a page and detennine the 
comment or write about a particular topic? Check all that apply. 

Response text 

I do not understand what is meant by authority in this question. 

I do not believe it is possible to determine the authority of a page returned by a search 
engine. 

I check to see if the author of the page has published other pages, articles or books about 
the topic. 

I look for infonnation about the author of the page - their qualifications, their degrees or 
certifications, their profession, their background, other pages/documents they have 
authored 

I look at the URL of the site, and based on the domain (.com, .edu, .org, .net, etc.) and 
use that information to help me detennine whether or not the site has authority. 

I do not examine the authority of a site 

I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

Evaluating the objectiveness of content 

When asked about the objectiveness of a Website in a TRUE/FALSE question (see Table 3), most 
respondents indicated they believed it was possible to evaluate the objectiveness of a Web page 
(74%), while a quarter (25%) indicated they believed it was not possible, and 1% did not understand 
the concept of objectiveness in relation to a Web page. 

Table 5 provides the results of a question concerning the evaluation of the objectiveness of an Internet 
source and reveals a more nuanced perspective on the part of the subjects. For this multiple-select 
question, 68% of respondents indicated that a fair and balanced discussion of the topic indicated the 
site was objective, a standard definition of the term, but an almost equal percentage (62%) indicated 


ability to 

Percentage 

17% 

6% 

43% 

43% 

39% 

10 % 

12 % 
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that they used the URL of the site and the domain to detennine whether or not the source was 
objective. Seven percent of respondents do not check the objectiveness of a site and an almost equal 
number (5%) do not believe it is possible to check the objectiveness of a site. Interestingly, 8% of the 
respondents felt that all pages returned by a search engine are objective. 


Table 5: Question 20 - How do you decide whether or not a source retrieved from the Internet is 
objective and provides fair and equal treatment of all sides of a topic? Check all that apply. 


Response text Percentage 

I do not understand what is meant by objective in this question 2% 

I believe that pages returned by my search engine are objective 8% 

I look at the URL of the site, and based on the domain (.com, .edu, .org, .net, etc.) and ^o/ 

use that information to help me detennine whether or not the site is objective. 

I check with someone who may know, for example library staff or a professor 25% 

I ask a friend if they think the site is objective. 8% 

If the document provides a fair discussion of all sides of a topic or issue and , 

acknowledges other viewpoints, then I consider it objective 

I do not evaluate the objectiveness of a site 7% 

I do not believe it is possible to detennine the objectiveness of a page returned by a ^ 

search engine 


Evaluation of content credibility 

When asked a TRUE/FALSE question about source credibility, approximately 80% of the subjects 
indicated that they felt it was possible to detennine the credibility of a Web page (see Table 3). 

When asked a multiple select question about how they determined the credibility of a document and 
detennined whether or not the document was trustworthy, subjects responded as shown in Table 6. 
Most subjects (57%) reported that they looked at the URL of the site and an equal percentage (58%) 
looked at who published the site. Approximately 9% believed all pages returned by a search engine 
are credible, 5% do not evaluate the credibility of a site and another 2% do not understand what is 
meant by credibility, so a total of 16% (9+5+2) of subjects in this sample appear to not be evaluating 
credibility of Web content. 


Table 6: Question 22 - How would you evaluate the credibility of a site or document and determine 
that the information is truthful and trustworthy? Check all that apply. 

Response text 

I do not understand what is meant by credibility in this question. 

I look at the URL of the site, and based on the domain (.com, .edu, .org, .net, etc.) and 
use that information to help me detennine whether or not the site is credible. 

I look at the background of the author of the page - their professional affiliations, their 
college degrees, what else they have written. 

I check to see who published the information on the site 
I believe that pages returned by my search engine are credible. 

I do not evaluate the credibility of Web sites. 


Percentage 

2% 

57% 

55% 

58% 

9% 

5% 
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I do not believe it is possible to evaluate the credibility of pages returned by a search 5% 

engine 

I evaluate the information on the site against what I know about the topic 36% 

Evaluation of content accuracy 

When asked a TRUE/FALSE question about the evaluation of the accuracy of Web content, over half 
(60 %) of the subjects indicated that they believed pages listed or returned by search engines usually 
contain accurate infonnation (see Table 3). 

When asked a multiple select question about how they evaluate the accuracy of infonnation on a 
page, subjects appeared to take a more conservative and nuanced view of search engine accuracy (see 
Table 7). Only 6% of the subjects indicated that search engine results are accurate (the word 'usually' 
was not used here, implying either pages were accurate or were not accurate). Over half of the 
respondents (60%) look at the URL to evaluate accuracy and an almost equal percentage (64%) 
evaluate the content on the site against their own prior store of knowledge (what they know about the 
topic). 


Table 7: Question 21 - How do you decide whether or not information on a page is accurate and 
contains truthful and correct information? Check all that apply. 

Response text 

I do not understand what is meant by accurate in this question 
I believe that pages returned by my search engine are accurate 

I look at the URL of the site, and based on the domain (.com, .edu, .org, .net, etc.) and 
use that information to help me detennine whether or not the site is accurate 
I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

I believe that sites with a more current date are more accurate 
I do not check the accuracy of information on a web site 

I evaluate the information on the page in relation to what I know about the topic 

I do not believe it is possible to detennine the accuracy of pages returned by a search 
engine 

I ask a friend if they think the page is objective 


Percentage 

2% 

6% 

60% 

34% 

30% 

5% 

64% 

5% 

4% 


Evaluation of page quality 

When asked about how they evaluate the quality of a Website, subjects responded as shown in Table 
8. Over half of the respondents (51%) appear to consider spelling and grammar to be part of the 
evaluation, and an equal percentage (51%) feel the URL of the site has some bearing on quality. Most 
(66%) of the subjects based quality on a determination of whether the page is comprehensive and 
covers the topic in considerable depth and over a third (38%) chose Web pages which were interesting 
and gave clear and well-reasoned explanations. 
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Table 8: Question 23 - How do you evaluate the quality of a page returned by a search engine? Check 

all that apply. 

Response text Percentage 

I do not understand what is meant by quality in this question. 

If the page includes pictures and charts it is a quality site 

If the page is free from spelling, typographical errors and the author uses good grammar, 

I consider it a quality site. 

If the infonnation presented in the site is comprehensive and covers the topic in 
considerable depth, I consider it a quality site. 

I look at the URL of the site, and based on the domain (.com, .edu, .org, .net, etc.) and 
use that information to help me detennine whether or not the site is a quality site. 

I do not evaluate the quality of pages returned by a search engine. 

If the infonnation on the page is interesting, and presents a clear, well-reasoned 
explanation of the topic, I consider it a quality page. 

I do not believe it is possible to determine the quality of a page returned by a search 
engine. 

I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

Evaluation of information sources in relation to years in college 

Evaluation of survey responses in relation to year in college provides some additional insights into the 
success or failure of current information literacy training efforts. Table 9 details the TRUE responses 
to a set of questions for early years college students (freshmen and sophomore year) and later years 
college students (junior and senior year). These results indicate a statistically significant change in 
students' perception of various properties of Internet sources in later years of college (X-squared = 
19.3077, df = 11, p-value = 0.056). Subjects appeared to be more circumspect in their evaluation of 
Web sources, with more subjects in later years indicating they could not identity the author of a Web 
page and more subjects indicating they could not determine the credibility of a Web page. 


3% 

11 % 

51 % 

66% 

51 % 

3% 

38% 

3% 

16% 


Table 9: Year in college and evaluation of sources questions. Percentage of TRUE responses (in- 



group percentages)> 



Question 

Question text 

Early Later 

14 

It is usually possible to detennine the author of a Web page 

18% 

14% 

15 

It is usually possible to detennine the qualifications of the author 

9% 

8% 

16 

It is usually possible to detennine whether or not a site is authoritative 

15% 

14% 

17 

It is usually possible to determine whether or not a page is objective 

20% 

21% 

18 

It is usually possible to detennine the credibility of a Web page 

20% 

24% 

19 

Web pages listed by a search engine usually contain accurate information. 

17% 

19% 


A question concerning the choice of information search tool also provides some indication of a 
change in the student's perception of sources in relation to year in college as indicated in Table 10. 
When asked whether they use a search engine or library database, survey respondents in later years of 
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college indicated a slightly greater preference for library databases and a slightly smaller preference 
for Google. These results are significant at the p <= 0.10 level (X-squared = 11.97, df = 7, p-value = 
0 . 10 ). 


Table 10: Question 13 - Which search engines or library databases do you use for 
your research? Check all that apply, (in-group percentages) 

Search engine/database Early (freshman-sophomore) Late (junior-senior) 


Yahoo 

6% 

4% 

Bing 

3% 

3% 

Google 

39% 

34% 

Other (Blekko/Lycos etc.) 

0% 

3% 

Cross-search engine aggregator 

2% 

1% 

General library 

27% 

29% 

Subject specific library 

13% 

17% 

Book catalogue 

9% 

8% 


Discussion 

The major findings from this study can be summarised as follows. 

• the Google search engine appears to be the most widely used source for academic research for 
subjects, though subjects did report using other sources 

• subjects had a high degree of confidence in search engine results, with the majority (60%) 
believing that pages returned by search engines usually contain accurate information 

• identification of the author of a Web page appeared to be difficult for subjects, with almost half 
(45%) indicating they could not determine the author of a Web page 

• evaluation of the authoritative characteristics of a source was difficult for subjects, with almost 
a quarter reporting they could not determine the authoritative characteristics of a site and almost 
a quarter reporting they did not understand the concept in that context 

• identification of the qualifications of the author were difficult for subjects, with a majority 
(63%) indicating they did not believe it was possible to evaluate author qualifications 

• a quarter (25%) of respondents did not consider it possible to determine the objectiveness of a 
Web page and over half (62%) used the URL and domain of the site to determine objectivity 

• subjects in later years of college (junior-senior) appear to be more discerning about Internet 
sources, less likely to use Google and more likely to use library databases 

These findings and others are discussed in more detail in the following paragraphs. 

Information search habits 

Google was the most common information source for this search sample, though respondents did 
provide some indication that they also commonly use other information sources. Nevertheless, this 
provides some quantitative confirmation of other findings which indicate that the Web and 
specifically Web content delivered through the Google commercial search engine continues to be a 
primary source for college level research (Head and Eisenberg, 2010 ; Biddix, Chung and Park, 2011) . 
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Evaluation of author qualifications and site authority 

Information literacy instruction implores students to ask questions about the authority of content 
sources, specifically to learn about the authors of the source. Students are asked to determine the 
author's credentials, reputation, knowledge about the subject, education and affiliation (Doyle and 
Hammond, 2006). For this sample, almost half (45%) of the subjects indicated they could not identify 
the author of a Web page and thus would not be able to meet that information literacy competency 
standard perfonnance outcome. This finding provides evidence to support Doyle and Hammond's 
(2006) proposal that identifying the source and determining authorship be supported through specific 
criteria (pp. 63-64). 

Similarly, when asked about the qualifications of the author of a Web page, a majority of respondents 
(63%) indicated they could not evaluate the qualifications of an author, providing indications of 
further issues concerning the evaluation of Web content authorship. Furthermore, in evaluating the 
authority of a site, a third of the respondents indicated they do not evaluate site authority for some 
reason, and 39% of respondents use the URL of the site to determine authority (perhaps not the best 
indicator of authority). Twelve percent confer with someone knowledgeable in the subject domain and 
43% check to see if the author has published other content. These results are an indication that 
subjects struggle with the identification of site authority using a variety of methods which may or may 
not provide good results. 

Evaluating objectiveness of content 

When teaching about objectivity, librarians expect that students will be able to identify if a source 
provides a fair discussion of all viewpoints. The majority of subjects in this study had confidence that 
they could evaluate the objectiveness of a Web page (74%). A majority of subjects (68%) attempted 
to determine if a site provides a fair discussion of all sides of a topic, an indication that they are 
making a valid effort to determine the objectiveness of content. Over half (62%) indicated they use 
the URL of the site to determine the objectivity of the source and a quarter of the subjects check with 
someone with more knowledge concerning a source. While a URL alone is not a good indicator of 
objectiveness, subjects were considering other criteria. That only 25% seek advice is potentially an 
indication that expediency may be more important to students than taking the time to perform a 
deeper evaluation of objectivity for a source. 

Evaluation of content credibility 

Most subjects indicated they could evaluate content credibility (80%), but 57% use the URL and 55% 
look at the author of the site, which almost half of the subjects indicated they could not identify. Five 
percent of subjects did not appear to be considering the credibility of a site at all. Additionally, about 
a third (36%) reviewed the site's content against what they already knew about the subject. This 
suggests that many students are not thinking critically about the information presented to them and 
evaluate credibility using a superficial and easy measure such as the site URL. This provides 
additional confirmation of a finding reported by other researchers that 'students value credibility but 
prefer efficiency' (Biddix, et al., 2011, p. 180). 

Evaluation of content accuracy 
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In response to a question about page selection, subjects indicated they used a somewhat balanced 
approach to selecting pages for their research, with a slight preference for understandability and 
currency of pages relative to other criteria. A majority of respondents believe the pages returned by 
search engines usually contain accurate information. The survey question asked if the results returned 
were usually accurate, so it is possible subjects allowed for inaccurate information sometimes being 
returned from search engines. Still this result appears to provide some indication that subjects were 
very confident in the accuracy of search engine results. This finding may be a disappointment to 
librarians who would prefer students not simply assume pages or articles are accurate. If students trust 
a search engine's results, giving credit to a source just because it appears in the first or second page of 
the results, as Head and Eisenberg (2010) also found, then there is little motivation to evaluate these 
sources. 

An additional question provided more in-depth examination of how subjects evaluate the accuracy of 
Internet sources. Survey responses indicated that approximately 60% of the subjects look at the URL 
to evaluate accuracy and 64% evaluate the content on the site against their own prior store of 
knowledge. Only about a third of the subjects look for help in evaluating the accuracy of a source, 
indicating that they were fairly confident they could evaluate the accuracy of the source through the 
URL or their own knowledge. 

Evaluation of content quality 

Survey respondents provided a somewhat balanced set of responses when asked about the evaluation 
of content quality. Half (51%) of the respondents considered the URL and domain to be an indicator 
of quality and an equal percentage considered the spelling and grammar on the site to be an indicator 
of quality. Most (66%) considered the depth and comprehensiveness of the site to be an indicator of 
quality and 38% considered the well-reasoned presentation of the topic to be the standard for site 
quality. This finding provides some indication that subjects are evaluating quality and using a number 
of different criteria, though it is debatable whether or not the URL and domain alone provide a clear 
indication of Web page quality. 

Evaluation of information sources in relation to years in college 

Survey respondents in later years of college (junior-senior) indicated a slight but statistically 
significant change in evaluation of Internet sources. Questions concerning the examination of various 
properties of a Web source revealed that subjects in later years of college were more aware that it may 
not be possible to determine the author of a Web page and the qualifications of the author. Subjects 
also indicated that they were less certain about the objectivity, authoritativeness and credibility of 
Web sources in later years of college. Conversely, these same subjects also appeared to be more 
certain of the accuracy of Web sources. 

This finding provides some indication that the junior and senior class students in this sample have 
learned some information literacy skills and report applying them in the evaluation of Web sources. 
However, there still appears to be a belief amongst these same students that Web sources are usually 
accurate sources. This certainty may reflect a personal confidence that they can determine the 
accuracy of a Web page or that they have been desensitised and falsely assume that this constant 
source of information (the Web) is truly accurate information. 
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Limitations of study 

Evaluation of the demographic questions in the survey revealed a slight bias towards women and 
students with a high grade point average (above 3.0). Additionally, the survey sample was from a 
single U. S. university. A slightly more diversified sample from more universities, domestic and 
international, might produce different results. 

Though subjects in this sample received information literacy instruction after taking the survey, it is 
possible that the subjects had previously received information literacy instruction in a previous year at 
the university or before coming to the university. 

Subjects may not have clearly understood the terms and concepts addressed in the questions in 
relation to information seeking. Many questions provided a response which could be used to indicate 
confusion with the concept, but not all questions provided this response. 

Conclusion 

The findings in this study add quantitative evidence to the assertion that the Web has become the 
primary source for research for college students. Though subjects gave some indication that other 
sources were used, the Web, through the lens of the commercial Google search engine, appears to be 
their primary research source. 

The goal of the study was to examine two research questions in relation to infonnation literacy 
standards for evaluation of Web sources. Evaluation of the sources was based on the criteria of 
authority, reliability, validity, accuracy, timeliness and point of view or bias. The first research 
question was whether searchers considered it possible to examine these criteria for documents found 
on the Web. Survey questions fourteen to nineteen addressed this research question. Our findings 
indicate that for the evaluative criteria of author, authoritativeness, objectivity, credibility and 
accuracy, the majority of subjects felt they could detennine these criteria for Web information 
sources. However, a majority of subjects felt they could not detennine the qualifications of the author 
of a Web page. The second research question was to detennine how searchers evaluate these criteria 
when examining information sources. Survey questions twenty to twenty-seven addressed this 
research question. Our findings indicate that most subjects reported using a number of different 
strategies to evaluate these criteria. While many of these strategies are logical and expected, careful 
examination of our results provides some indication of confusion amongst the subjects about how to 
perform these evaluations. These findings are discussed in more detail in the following paragraphs. 

While most subjects (60%) responding to a TRUE/FALSE question concerning search engine 
accuracy indicated they believed search engine results are usually accurate, responses to more detailed 
questions concerning search engine results were more nuanced. The results of a series of questions 
asking survey respondents about credibility, quality and authority of search engine results indicated 
students examine a number of different criteria in evaluating sources. This could be an indication that 
a decade of information literacy instruction and expressions of concern by university faculty have had 
an impact on students' perceptions of search engine results. Still, the indication that most subjects 
considered search engine results usually accurate should be a concern. Given that students and 
instructors understand little of the unpublished algorithms which drive commercial search engines, it 
cannot be assumed that results returned by these applications meets any of the Association of College 
and Research Libraries information literacy competence evaluative criteria, including the criteria of 
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accuracy. Though Google's search engine algorithm is proprietary and not publicly available, 
evidence available indicates that Google's search engine results are ordered largely by number of li nk s 
and user selection counts, with no measure of accuracy or quality (Brin and Page, 19981 . It is 
important that students understand this and evaluate the documents returned by Google accordingly. 

Information literacy instruction emphasises identification of the author. This is crucial in 
understanding the authority or ability of the author to write about a particular topic. There is perhaps 
no other area where the distinction between traditional publishing and the Web creates difficulties for 
researchers using Web sources. The nature of the free-fonn publishing of the Web makes it very 
difficult to detennine the true authorship of a page. The technology of the Web makes the 
development of collaborative documents extremely easy, leading to sites such as Wikipedia. While 
debate on the quality of the content of a collaborative encyclopaedic site such as Wikipedia is beyond 
the scope of this paper, the authorship of the content and the qualifications of the authors are clearly 
difficult to ascertain. Information literacy instruction and course instructors should address this issue. 

While subjects provided various answers in response to the question about evaluating the 
objectiveness of a Web page, the indication that subjects use the URL or site domain to determine 
objectivity is problematic. Students should be more discerning and information literacy instruction 
should encourage more careful evaluation of objectivity of Web page sources. 

Subjects in this study used a variety of techniques to evaluate Web content. In examining their 
evaluation in relation to Association of College and Research Libraries information literacy standards, 
subjects appeared to follow some standards but were confused about how to actually implement them. 
Specifically, identifying content authors, assigning authority for content and determining accuracy 
and objectivity appeared difficult for subjects. While subjects did indicate that they were attempting 
to perform these actions and did appear to have confidence in their ability to perform these actions, 
their selection of methods was inconsistent and problematic. These could be indications of problems 
with information literacy instruction standards or broader problems with assigning authorship and 
detennining authority and accuracy of Web content in general. 

The results of this study provide some indication of flaws in a Website evaluation checklist 
instructional model. Additionally, the broad proposals on evaluating information sources in the 
Association of College and Research Libraries information literacy standards may require context- 
sensitive strategies which are specific to various fields. Librarians alone cannot remedy this, since 
librarians cannot be experts in all fields. Faculty and librarian partnerships in course instruction may 
provide a solution. This can be accomplished using an embedded librarian within the course to 
accomplish more in helping students to identify authors, authority, accuracy, objectivity, point of 
view and other important criteria to detennine which sources are really the most appropriate to use in 
the context of the course and assignment. 

Future directions 

The Association of College and Research Libraries has recognised that the Web and college students 
have changed since the release of the original infonnation literacy competency standards in 2000 and 
they have been 'rethinking the standards' (Bell, 20 13) . Kuhlthau (20131 highlights how the current 
standards are not holistic and do not adequately prepare students to research reflectively in the 
'dynamic global information environment' (p. 97). 
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The Association of College and Research Libraries Information Literacy Standards Task Force has 
proposed a Framework for Information Literacy. The framework explores information literacy criteria 
with threshold concepts (core understandings), a set of abilities or knowledge practices, and related 
metaliteracy objectives. These objectives expand evaluation criteria definitions and encourage 
evaluation of context referred to as the 'information need of the moment' (Gibson and Jacobson, 

2014) . 

The results of this study demonstrate the need for course instructors to engage with information 
literacy objectives through their semester long courses. The new Association of College and Research 
Libraries information literacy framework provides a structure for doing this. Librarians need to 
partner with course instructors to design assignments that encourage students to think critically about 
the selection of information sources relating to course content. 

Subject domains cannot be ignored given the nature of the information available on the Web, the lack 
of access mediation and the relative lack of peer or expert review of much of the content available. 

The evaluation of the results from these distinct domains may require a different set of standards and 
rules for each. Some subject-specific information literacy standards have been compiled by the 
American Library Association, but not all disciplines are currently represented (see Information 
Literacy Standards Committee, 2013 for the list of disciplines with subject specific standards). 
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Appendices 

Appendix A - Survey questions 

1. If you have declared a major or completed work for a college undergraduate degree, what is or 
was th earea of study for your declared undergraduate major? If you have multiple declared 
majors or multiple degrees, check all that apply. 

a. I have not declared a major 

b. business 

c. education 

d. sciences 

e. communication 

f. arts 

g. music 

h. math 

i. oreign language 

j. English language 

k. journalism 

l. theatre 

m. nursing 

n. engineering 

o. my major is not listed 

2. What is your gender? 

a. male 
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b. female 

3. What is your age? 

a. number range from 18 to 85 

4. Is English your first language? 

a. yes 

b. no 

5. How often do you use the Web to find information? 

a. several times a day 

b. about once a day 

c. about once a week 

d. about once a month 

6. How many years of college have you completed? 

a. How many years of college have you completed? 

b. Sophomore 

c. Junior 

d. Senior 

e. Graduate 

f. Masters 

g. Graduate - Ph.D. 

7. How many papers have you written in the last year that required research, evaluation and 
citation of sources? 

a. 0 

b. 1 

c. 2 

d. 3 

e. 4 or more 

8. What is your current GPA? 

a. below 2.0 

b. above 2.0 and below 2.5 

c. 2.5 and above and below 3.0 

d. 3.0 and above and below 3.5 

e. 3.5 and above 

f. not applicable or do not know 

9. When a search engine returns a list of pages, I select a page from the list based on the following 
criteria. Check all that apply. 

a. the fonnat of the site 

b. whether or not the site has video 

c. the site has pictures 

d. the site has a lot of infonnation 

e. the site is understandable 

f. I am familiar with the author of the site 

g. I can verify the information on the site 

h. the credentials (qualifications) of the author are good 

i. the site is current 

j. the quality of the writing on the site is good 

k. there are reviews and comments on the site 

10. Consider the papers that you have written in the last year. How often have you used research 
tools beyond Google or Ask.com to do your research? 
a. Never 
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b. Infrequently - less than 25 per cent of the time. 

c. Sometimes - about 50 per cent of the time. 

d. Usually - about 75 per cent of the time. 

e. Almost always 

11. How would you know when you have enough sources for a paper? 

a. I don't worry about the number of sources for a paper. 

b. 5 sources is enough for any paper 

c. 10 sources is enough for any paper 

d. 20 sources is enough for any paper 

e. I try to find enough quality sources to support the information in my paper. 

12. 12. Consider the papers that you have written in the last year. How many pages of search 
engine (Google, Yahoo, Bing) results do you usually view? 

a. 1 page 

b. 2 pages 

c. more than 2 pages 

d. more than 5 pages 

13. Which search engines or library databases do you use for your research? Check all that apply. 

a. Yahoo! 

b. Bing 

c. Google 

d. Other Search engine such as: Blekko/Lycos/AOL 

e. Metasearch engine (one that cross searches other search engines) such as 
Ixquick/Dogpile/Clusty/Webcrawler/Mywebsearch 

f. General Library databases such as Academic Search Premier/CQ Researcher/Credo 
Reference/LexisNexis/JSTOR/Omnifile/Gale Virtual Reference /Ebscohost 

g. Subject specific library databases such as: ABI-Infom, ArtStor, Business Source Premier, 
Communication 

h. Library OneSearch (cross-searches the library catalog and most of our databases) 

i. Library Book Catalog 

14. It is usually possible to determine the author of a web page. 

a. TRUE 

b. FALSE 

15. It is usually possible to determine the qualifications of the author of a web page. 

a. TRUE 

b. FALSE 

c. I do not understand what is meant by qualifications in this question 

16. It is usually possible to determine whether or not a site is authoritative. 

a. TRUE 

b. FALSE 

c. I do not understand what is mean by authoritative in this question 

17. It is usually possible to determine whether or not a web page is objective and unbiased. 

a. TRUE 

b. FALSE 

c. I do not understand what is meant by objective and unbiased 

18. It is usually possible to determine the credibility of a web page. 

a. TRUE 

b. FALSE 

c. I do not understand the meaning of credibility in this question 

19. I believe the pages listed by a search engine results usually contain accurate infonnation. 
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a. TRUE 

b. FALSE 

20. How do you decide whether or not a source retrieved from the Internet is objective and provides 
fair and equal treatment of all sides of a topic? Check all that apply. 

a. I do not understand what is meant by objective in this question 

b. I believe that pages returned by my search engine are objective 

c. I look at the URL of the site, and based on the domain (.com,., .edu, .org, .net, etc.) and 
use that information to help me determine whether or not the site is objective. 

d. I check with someone who may know, for example library staff or a professor 

e. I ask a friend if they think the site is objective. 

f. If the document provides a fair discussion of all sides of a topic or issue and 
acknowledges other viewpoints, then I consider it objective 

g. I do not evaluate the objectiveness of a site 

h. I do not believe it is possible to determine the objectiveness of a page returned by a 
search engine 

21. How do you decide whether or not infonnation on a page is accurate and contains truthful and 
correct information? Check all that apply. 

a. I do not understand what is meant by accurate in this question 

b. I believe that pages returned by my search engine are accurate 

c. I look at the URL of the site, and based on the domain (.com,.edu, .org, .net, etc.) and use 
that information to help me determine whether or not the site is accurate 

d. I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

e. I believe that sites with a more current date are more accurate 

f. I do not check the accuracy of infonnation on a web site 

g. I evaluate the infonnation on the page in relation to what I know about the topic 

h. I do not believe it is possible to determine the accuracy of pages returned by a search 
engine 

i. I ask a friend if they think the page is objective 

22. How would you evaluate the credibility of a site or document and detennine that the 
infonnation is truthful and trustworthy? Check all that apply. 

a. I do not understand what is meant by credibility in this question. 

b. I look at the URL of the site, and based on the domain (.com,.edu, .org, .net, etc.) and use 
that information to help me determine whether or not the site is credible. 

c. I look at the background of the author of the page - their professional affiliations, their 
college degrees, what else they have written. 

d. I check to see who published the infonnation on the site 

e. I believe that pages returned by my search engine are credible. 

f. I do not evaluate the credibility of web sites. 

g. I do not believe it is possible to evaluate the credibility of pages returned by a search 
engine 

h. I evaluate the information on the site against what I know about the topic 

23. How do you evaluate the quality of a page returned by a search engine? Check all that apply. 

a. I do not understand what is meant by quality in this question. 

b. If the page includes pictures and charts it is a quality site 

c. If the page is free from spelling, typographical errors and the author uses good grammar, 

I consider it a quality site. 

d. If the information presented in the site is comprehensive and covers the topic is 
considerable depth, I consider it a quality site. 
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e. I look at the URL of the site, and based on the domain (.com,.edu, .org, .net, etc.) and use 
that information to help me determine whether or not the site is a quality site. 

f. I do not evaluate the quality of pages returned by a search engine. 

g. If the information on the page is interesting, and presents a clear, well-reasoned 
explanation of the topic, I consider it a quality page. 

h. I do not believe it is possible to determine the quality of a page returned by a search 
engine. 

i. I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

24. How do you evaluate the authority of a page and determine the ability to comment or write 
about a particular topic? Check all that apply. 

a. I do not understand what is meant by authority in this question. 

b. I do not believe it is possible to determine the authority of a page returned by a search 
engine. 

c. I check to see if the author of the page has published other pages, articles, or books about 
the topic. 

d. I look for information about the author of the page - their qualifications, their degrees or 
certifications, the profession, their background, other pages/documents they have 
authored 

e. I look at the URL of the site, and based on the domain (,com,.edu, .org, .net, etc.) and use 
that information to help me determine whether or not the site has authority. 

f. I do not examine the authority of a site 

g. I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

25. How do you evaluate the currency of a page and determine how current the information on the 
site is? Check all that apply. 

a. I do not understand what is meant by currency in this question. 

b. I examine how frequently the site has been updated. 

c. I look for a date on the site 

d. I do not believe it is possible to determine if the infonnation returned by a search engine 
is current. 

e. I do not evaluate the currency of a site 

f. I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

g. I look for a copyright date 

h. I look at the dates of the references 

i. I check to see if all the li nk s are still active (broken links will lead me to believe the site 
is not current) 

26. How do you evaluate the purpose or bias of a web site? Check all that apply. 

a. I do not understand what is meant by purpose in this question. 

b. I detennine whether or not the author of the page or the owner of the URL is trying to sell 
something. 

c. I examine whether or not the purpose of the site is to promote a particular opinion or 
point of view 

d. I examine whether or not the site is a spam, hoax or joke 

e. I do not evaluate the purpose of a site. 

f. I do not believe it is possible to determine the purpose of a page returned by a search 
engine. 
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g. I check with someone with knowledge of the site or topic, for example, library staff or a 
professor 

27. How do you evaluate the relevance of a page returned by a search engine? Check all that apply. 

a. I do not understand what is meant by relevance in this question 

b. I detennine whether the page is useful for the research I am doing 

c. I can understand the information on the page 

d. I detennine whether or not the information on the page adds to my knowledge on the 
topic 

e. I do not believe it is possible to determine the relevance of a page returned by a search 
engine. 


Appendix B - Survey questions and Association of College and Research 
Libraries information literacy competency standards for higher education (2000) 

Questions 1-8 are demographic and not mapped to standards. 

ACRL IL competency standard/performance 

Question in survey instrument indicator outcomes from (Association of College 

and Research Libraries, 2000) 

Question 9: When a search engine returns a 
list of pages, I select a page from the list based 

on the following criteria. Check all that apply. 1,2c: The information literate student identifies the 

value and differences of potential resources in a 

a. the format of the site variety of formats (e.g., multimedia, database, 

b. whether or not the site has video website, data set, audio/visual, book) 

c. the site has pictures 

d. the site has a lot of information 2.4a: The information literate student assesses the 

e. the site is understandable quantity, quality, and relevance of the search results 

f. I am familiar with the author of the site to detennine whether alternative information retrieval 

g. I can verify the infonnation on the site systems or investigative methods should be utilised 

h. the credentials (qualifications) of the 

author are good 3.2a: The information literate student examines and 

i. the site is current compares infonnation from various sources in order 

j. the quality of the writing on the site is to evaluate reliability, validity, accuracy, authority, 

good timeliness, and point of view or bias 

k. k.) there are reviews and comments on 
the site 

Question 10: Consider the papers that you 

have written in the last year. How often have 2.2e: The infonnation literate student implements the 
you used research tools beyond Google or search strategy in various infonnation retrieval 
Ask.com to do your research? systems using different user interfaces and search 

engines, with different command languages, 
protocols, and search parameters. 
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2.3: The infonnation literate student retrieves 
infonnation online or in person using a variety of 
methods. 

2.3a: The infonnation literate student uses various 
search systems to retrieve information in a variety of 
fonnats. 


1: The information literate student determines the 
nature and extent of the information needed. 

Question 11: How would you know when you 

have enough sources for a paper? 3.4 a: The information literate student detennines 

whether infonnation satisfies the research or other 
infonnation needed. 


1: The information literate student determines the 
nature and extent of the information needed. 


Question 12: Consider the papers that you 
have written in the last year. How many pages 
of search engine (Google, Yahoo, Bing) 
results do you usually view? 


3.4 a: The information literate student detennines 
whether information satisfies the research or other 
infonnation need. 

3.7c: The infonnation literate student reviews 
infonnation retrieval sources used and expands to 
include others as needed. 


Question 13: Which search engines or library 
databases do you use for your research? Check 
all that apply. 


2.Id: The information literate student selects efficient 
and effective approaches for accessing the 
infonnation needed from the investigative method or 
infonnation retrieval system. 


a. Yahoo 

b. Bing 

c. Google 

d. Other search engine 

e. Metasearch engine 

f. General library database 

g. Subject specific library database 

h. Library one search 

i. Library book catalogue 


2.2e: The infonnation literate student implements the 
search strategy in various infonnation retrieval 
systems using different user interfaces and search 
engines, with different command languages, 
protocols, and search parameters. 

2.3: The infonnation literate student retrieves 
infonnation online or in person using a variety of 
methods. 


Question 14: It is usually possible to 

determine the author of a Web page. 3.2a: The infonnation literate student examines and 

compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 
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3.4e: The infonnation literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 


Question 15: It is usually possible to 
determine the qualifications of the author of a 
Web page. 

Question 16: It is usually possible to 
determine whether or not a site is 
authoritative. 


3.4e : The information literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions 

3.2a: The infonnation literate student examines and 
compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 


Question 17: It is usually possible to 
determine whether or not a Web page is 
objective and unbiased. 


1.2d: The infonnation literate student identifies the 
purpose and audience of potential resources. 

3.2a: The infonnation literate student examines and 
compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 

3.4b: The infonnation literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verifies information used 
from other sources. 


3.2a: The infonnation literate student examines and 
compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 

3.4b: The infonnation literate student uses 

„ . * . ,, A consciously selected criteria to detennine whether the 

Question 18: It is usually possible to . „ . J _ . , 

' . , mlonnation contradicts or verities information used 

determine the credibility ot a Web page. . 

r Irom other sources. 

3.4e: The infonnation literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 

Question 19:1 believe the pages listed in 

search engine results usually contain accurate 3.2a: The infonnation literate student examines and 
information. compares infonnation from various sources in order 
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to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 

3.4b: The infonnation literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verities information used 
from other sources. 

3.4e: The infonnation literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 


Question 20: How do you decide whether or 
not a source retrieved from the Internet is 
objective and provides fair and equal 
treatment of all sides of a topic? Check all that 
apply. 

a. I do not understand what is meant by 
objective in this question 

b. I believe that pages returned by my 
search engine are objective 

c. I look at the URL of the site, and based 
on the domain (.com, .edu, .org, .net, 
etc.) use that information to help me 
determine whether or not the site is 
objective 

d. I check with someone who may know, 
for example library staff or a professor 

e. I ask a friend if they think the site is 
objective. 

f. If the document provides a fair 
discussion of all sides of a topic or issue 
and acknowledges other viewpoints, 
then I consider it objective 

g. I do not evaluate the objectiveness of a 
site 

h. I do not believe it is possible to 
determine the objectiveness of a page 
returned by a search engine 


3.2: The infonnation literate student articulates and 
applies initial criteria for evaluating both the 
information and its sources. 


a. Examines and compares information from 
various sources in order to evaluate reliability, 
validity, accuracy, authority, timeliness, and 
point of view or bias 

b. analyses the structure and logic of supporting 
arguments or methods 

c. Recognises prejudice, deception, or 
manipulation 

d. Recognises the cultural, physical, or other 
context within which the information was 
created and understands the impact of context 
on interpreting the information 


3.4e The information literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 


3.4b: The infonnation literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verifies information used 
from other sources. 


Outcomes include: 


3.6: The infonnation literate student validates 
understanding and interpretation of the information 
through discourse with other individuals, subject-area 
experts, and/or practitioners. 
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3.6c: The infonnation literate student seeks expert 
opinion through a variety of mechanisms (e.g., 
interviews, email, listservs). 


Question 21: How do you decide whether or 
not infonnation on a page is accurate and 
contains truthful and correct information? 
Check all that apply. 


3.2 The infonnation literate student articulates and 
applies initial criteria for evaluating both the 
information and its sources. 


a. 


b. 


c. 


d. 


e. 


f. 

g- 

h. 


I do not understand what is meant by 
accurate in this question 
I believe that pages returned by my 
search engine are accurate 
I look at the URL of the site, and based 
on the domain (.com, .edu, .org, .net, 
etc.) use that information to help me 
determine whether or not the site is 
accurate 

I check with someone with knowledge 
of the site or topic, for example, library 
staff or a professor 

I believe that sites with a more current 
date are more accurate 
I do not check the accuracy of 
information on a Website 
I evaluate the information on the page in 
relation to what I know about the topic 
I do not believe it is possible to 
determine the accuracy of pages 
returned by a search engine 
I ask a friend if they think the page is 
accurate 


3.2a: The infonnation literate student examines and 
compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 

3.4: The infonnation literate student compares new 
knowledge with prior knowledge to determine the 
value added, contradictions, or other unique 
characteristics of the infonnation. 

3.4b: The infonnation literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verities information used 
from other sources. 

3.4g: The information literate student selects 
infonnation that provides evidence for the topic. 

3.6c: The infonnation literate student seeks expert 
opinion through a variety of mechanisms (e.g., 
interviews, email, listservs). 


Question 22: How would you evaluate the 
credibility of a site or document and 
determine that the infonnation is truthful and 
trustworthy? Check all that apply. 


3.2a: The infonnation literate student examines and 
compares infonnation from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 


a. 


b. 


c. 


I do not understand what is meant by 
credibility in this question. 

I look at the URL of the site, and based 
on the domain (.com, .edu, .org, .net, 
etc.) use that information to help me 
determine whether or not the site is 
credible. 

I look at the background of the author of 
the page - their professional affiliations, 


3.4b: The infonnation literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verifies information used 
from other sources. 

3.4e: The infonnation literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 
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their college degrees, what else they 
have written. 

d. I check to see who published the 
information on the site 

e. I believe that pages returned by my 
search engine are credible. 

f. I do not evaluate the credibility of 
Websites. 

g. I do not believe it is possible to evaluate 
the credibility of pages returned by a 
search engine 

h. I evaluate the information on the site 
against what I know about the topic 


Question 23: How do you evaluate the 
quality of a page returned by a search engine? 
Check all that apply. 

a. I do not understand what is meant by 
quality in this question. 

b. If the page includes pictures and charts 
it is a quality site 

c. If the page is free from spelling, 
typographical errors and the author uses 
good grammar, I consider it a quality 
site. 

d. If the infonnation presented in the site is 
comprehensive and covers the topic in 
considerable depth, I consider it a 
quality site. 

e. I look at the URL of the site, and based 
on the domain (.com, .edu, .org, .net, 
etc.) use that information to help me 
determine whether or not the site is a 
quality site. 

f. I do not evaluate the quality of pages 
returned by a search engine. 

g. If the infonnation on the page is 
interesting, and presents a clear, well- 
reasoned explanation of the topic, I 
consider it a quality page. 

h. I do not believe it is possible to 
determine the quality of a page returned 
by a search engine. 

i. I check with someone with knowledge 
of the site or topic, for example, library 
staff or a professor 


3.2: The infonnation literate student articulates and 
applies initial criteria for evaluating both the 
information and its sources. 

3.4: The infonnation literate student compares new 
knowledge with prior knowledge to determine the 
value added, contradictions, or other unique 
characteristics of the infonnation. 

3.4b: The information literate student uses 
consciously selected criteria to detennine whether the 
infonnation contradicts or verities information used 
from other sources. 

3.4e: The infonnation literate student determines 
probable accuracy by questioning the source of the 
data, the limitations of the information gathering 
tools or strategies, and the reasonableness of the 
conclusions. 
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Question 24: How do you evaluate the 
authority of a page and determine the ability 
to comment or write about a particular topic? 

Check all that apply. 

a. I do not understand what is meant by 
authority in this question. 

b. I do not believe it is possible to 
determine the authority of a page 
returned by a search engine. 

c. I check to see if the author of the page 
has published other pages, articles or 
books about the topic. 

d. I look for infonnation about the author 
of the page - their qualifications, their 
degrees or certifications, their 
profession, their background, other 
pages/documents they have authored 

e. I look at the URL of the site, and based 
on the domain (.com, .edu, .org, .net, 
etc.) use that information to help me 
determine whether or not the site has 
authority. 

f. I do not examine the authority of a site 

g. I check with someone with knowledge 
of the site or topic, for example, library 
staff or a professor 

Question 25: How do you evaluate the 3.2: The infonnation literate student articulates and 

currency of a page and detennine how current applies initial criteria for evaluating both the 
the information on the site is? Check all that information and its sources, 
apply 

3.2a: The infonnation literate student examines and 

a. I do not understand what is meant by compares infonnation from various sources in order 

currency in this question. to evaluate reliability, validity, accuracy, authority, 

b. I examine how frequently the site has timeliness, and point of view or bias, 
been updated. 

c. I look for a date on the site 3.6c: The infonnation literate student seeks expert 

d. I do not believe it is possible to opinion through a variety of mechanisms (e.g., 

determine if the information returned by interviews, email, listservs). 

a search engine is current 

e. I do not evaluate the cunency of a site 

f. I check with someone with knowledge 
of the site or topic, for example, library 
staff or a professor 

g. I look for a copyright date 

h. I look at the dates of the references 
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i. I check to see if all the li nk s are still 
active (broken links will lead me to 
believe the site is not current) 


How do you evaluate the purpose or bias of a 
Website? Check all that apply. 


a. 

b. 


c. 


d. 

e. 

f. 

g- 


I do not understand what is meant by 
purpose in this question. 

I determine whether or not the author of 
the page or the owner of the URL is 
trying to sell something. 

I examine whether or not the purpose of 
the site is to promote a particular 
opinion or point of view 
I examine whether or not the site is a 
spam, hoax or joke 

I do not evaluate the purpose of a site. 

I do not believe it is possible to 
determine the purpose of a page 
returned by a search engine. 

I check with someone with knowledge 
of the site or topic, for example, library 
staff or a professor 


1.2d: Identifies the purpose and audience of potential 
resources (e.g., popular vs. scholarly, current vs. 
historical). 

3.2a: The information literate student examines and 
compares information from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 

3.2c: The information literate student recognises 
prejudice, deception, or manipulation. 


Question 27: How do you evaluate the 
relevance of the documents you retrieve from 
the Web? 


a. I do not understand what is meant by 
relevance in this question 

b. I determine whether the page is useful 
for the research I am doing 

c. I can understand the information on the 
page 

d. I determine whether or not the 
information on the page adds to my 
knowledge on the topic 

e. I do not believe it is possible to 
determine the relevance of a page 
returned by a search engine. 


3.2a: The information literate student examines and 
compares information from various sources in order 
to evaluate reliability, validity, accuracy, authority, 
timeliness, and point of view or bias. 
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